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[57] ABSTRACT 

A method and apparatus for splicing a first compressed 
digital information stream into a second compressed digital 
information stream. The first information stream includes at 
least one entrance indicium identifying an appropriate 
stream entrance point, the second information stream 
includes at least one exit indicium identifying an appropriate 
stream exit point. A controller monitors the two streams until 
the appropriate points are found and, in response to a control 
signal, splices the first stream into the second stream. 

16 Claims, 6 Drawing Sheets 
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INFORMATION STREAM SYNTAX FOR stream. The invention monitors the two streams until the 

INDICATING THE PRESENCE OF A SPLICE appropriate points are found and, in response to a control 

POINT signal, splices the first stream into the second stream. 

Specifically, the inventive splicer includes a pre-splice 

This application is a continuation in part of U.S. patent 5 buffer receiving a first information stream and producing a 
application Ser. No. 08/864,322, filed on May 28, 1997 for buffered information stream; a bitstream examiner receiving 

a METHOD AND APPARATUS FOR SPLICING COM- the first information stream and responsively causing the 

PRESSED INFORMATION STREAMS, which is herein pre-splice buffer to position an entrance point of the buffered 

incorporated by reference. information stream at an output of the buffer; a switch for 

This invention was made with U.S. government support io coupling either the buffered information stream or a second 

under contract number 70NANB5H1174. The U.S. Govern- information stream to an output; and a switch controller for 

ment has certain rights in this invention. monitoring the second information stream and, in response 

The invention relates to communication systems in to a control signal and the detection of an exit point in the 

general, and more particularly, the invention relates to a second information stream, causing the switch to couple the 

method for identifying and utilizing splicing "in-points" and 15 buffered information stream to an output, 
splicing "out-points" in MPEG-like information stream. 

BACKGROUND OF THE DISCLOSURE 

The teachings of the present invention can be readily 

In several communications systems, the data to be trans- under stood by considering the following detailed descrip- 

mitted ^compressed so that the available bandwidth is used Uon in conjunction with the accompany in g drawings, in 

more efficiently. For example, the Moving Pictures Experts which* 

Group (MPEG) has promulgated several standards relating ' , t , , e JX . 

to digital data delivery systems. The first, known as MPEG-1 , FIG * 1 shows a b °^ di: f am of a com P re s* d bitstream 

refers to ISO/IEC standards 11172, incorporated herein by s P hcm § s y stem includm S the inventlon i 

reference. The second, known as MPEG-2, refers to ISO/ 25 FIG - 2 depicts a flow chart of a seamless splicing process 

IEC standards 13818, incorporated herein by reference. A in accordance with the invention; 

compressed digital video system is described in the FIG. 3 shows a detailed block diagram of the splicer of 

Advanced Television Systems Committee (ATSC) digital FIG. 1; 

television standard document A/53, incorporated herein by FIG. 4 depicts a block diagram of digital studio compris- 

reference. m g a plurality of interoperable islands and including the 

A program transport stream is formed by multiplexing invention; 

individual elementary streams which share a common time FIGS. 5A-5C depicts a plurality of splicing scenarios; 

base (i.e., the same 27 MHz clock source). The elementary FIG. 6 depict a flow diagram of a routine 600 suitable for 

streams comprise encoded video audio or other bit streams. 35 identifying ia .p oints an d out-points in accordance with the 

The elementary streams may be, but do not have to be, in a invention 

packetized elementary stream (PES) format prior to trans- _ - ' , .. ■ , • * r 

port multiplexing. A PES consists of a packet header fol- t To ? acilltate understanding, identical reference numerals 

lowed by a packet payload. As the elementary streams are h f ve becn _ used ' wherc P™^: t0 designate identical 

multiplexed, they are formed into transport packets and a 4Q elements that are mmma to the ^ wcs - 

control bit stream that describes the program (also formed DETAILED DESCRIPTION 

into transport packets) is added. 

There are many instances where there is a need to switch The invention is generally described within the context of 
from one encoded or compressed bitstream to another. When a digital television studio includes a plurality of operative 
switching from one compressed MPEG video bitstream to 45 environments which receive and process various bitstreams 
another, appropriate measures must be taken in the trans- and which have associated switching capabilities according 
mission order of the picture bitstream to assure proper to the invention. The switching capabilities allow seamless 
subsequent presentation of the decoded pictures, without or non-seamless splicing of a plurality of, e.g., video trans- 
time gaps. Such time gaps result in undesirable video or port streams to produce an output stream. A combination of 
audio artifacts (e.g., blank screen due to buffer overflow/ 50 seamless and non -seam less bitstreams may be produced to 
underflow, poor "lip sync" and the like). Heretofore there provide a controllably degraded output stream, 
has not existed a seamless splicing method and apparatus for The invention is a two-input bitstream splicer which 
splicing transport streams to one another. performs switching, splicing or insertion operations on a 

Therefore, a need exists in the art for a method and pair of MPEG-compliant input transport streams to produce 

apparatus for splicing compressed digital information bit- 55 an output stream. It must be noted that the principles of the 

streams. invention apply to bitstream switchers or splicers having 

more than two inputs and to input streams other than 

SUMMARY OF THE INVENTION MPEG-compliant input streams. The invention may be 

The disadvantages heretofore associated with the prior art implemented using a general purpose computer system that 
are overcome by the present invention of a method and 60 ^ programmed to perform the functions discussed below. As 
apparatus for splicing compressed digital information programmed, the general purpose computer becomes a spe- 
streams. In particular, the invention splices a first informa- cific purpose apparatus for splicing digital data bit streams, 
tion stream into a second information stream. The first The invention may be used for both seamless and non- 
information stream includes at least one entrance indicium seamless splicing of bitstreams. Seamless splicing means 
that identifies an appropriate point of entrance to the stream. 65 seamless butt-splicing of two streams to form a resultant 
The second information stream includes at least one exit output stream that produces a continuous, undisturbed flow 
indicium that identifies an appropriate point of exit from the of information (e.g., video or audio without glitches or 
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artifacts). Non-seamless splicing produces a resultant output 
signal which may have a disturbed information flow (e.g., 
visual or aural distortions, disturbances and artifacts). For 
purposes of this discussion, it will be assumed that each 
bitstream is a transport stream comprising video, audio and 
(possibly) other information. It must be noted that the 
invention is applicable to packetized elementary and other 
elementary streams. Additionally, it is assumed that the 
splicing points are determined with respect to the video 
information. This may result in some distortions in the 
spliced audio and other information, since the audio and 
other information may not temporally "line up" on a packet 
by packet basis. 

Splicing consists of making a transition in an output- 
stream from a "from-stream" to a "to -stream." The from- 
stream is ideally exited at an "out-point" and the to-stream 
is ideally entered at an "in -point." An out-point is a place in 
a presently-selected stream (i.e., "from-stream") where the 
stream may be ended, and some other stream (i.e., "to- 
stream") spliced on. An "in-point" is a place in the other 
stream where the information may begin to be spliced on to 
another stream. 

A "splicing segment" is defined as the portion of an 
information stream between an in-point and an out-point. A 
splicing segment may include multiple out -points and 
in-points. Thus, it is desirable to include as many in-points 
and out-points as possible in a stream to allow for maximum 
flexibility in splicing. Within the definition of an in-point 
and an out -point is a delay-parameter, e.g., a video buffering 
verifier (VBV) for MPEG compliant streams. A splicing 
segment with a known in-point delay-parameter and with 
out-points having the same known delay-parameter may 
include within itself shorter valid splicing segments with 
different values of the delay-parameter. 

In the context of a studio environment of an exemplary 
embodiment of the invention, information streams are 
divided into transport packets. Packets containing video may 
be intermixed with packets containing audio, auxiliary data, 
or other information. In this environment, a video stream 
out-point is the end of the last video transport packet of the 
stream of interest. The video stream before and through the 
last packet must meet the splicing definition of an out-point. 
Similarly, a video stream in-point is the beginning of the first 
video transport packet of a splice segment (SS). It must be 
noted that other information in the transport stream, specifi- 
cally audio, is unlikely to be neatly segmented at in-points 
and out-points. A method for correcting errors induced by 
the non-alignment of audio transport packets is described in 
U.S. patent application Ser. No. 08/864,321, filed May 28, 
1997, incorporated herein by reference. 

A critical aspect of splicing information streams is the 
proper processing of the various delay parameters. One 
parameter of concern is the delay parameter associated with 
the various information streams. In the case of an MPEG- 
compliant stream, the delay parameter is the video buffering 
verifier (VBV) delay parameter. Another parameter is the 
latency, or transitional period, inherent in a splicing opera- 
tion. For example, a typical splice occurs at a certain time, 
i.e., a "splice time." Prior to the splice time an output 
information stream comprises a from -stream. At the splice 
time, a switch to the to-stream occurs. For a period of time 
the output stream may include information from both the 
from -stream and the to-stream. Eventually the output stream 
includes information from only the to-stream. 

It is assumed that the from -stream and the to-stream are 
each valid. There are certain constraints on the streams that 
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must be met if the splicing is to be seamless. Seamless 
splicing implies that the resultant spliced bitstream will not 
cause discontinuities in the future. 

One specific example of a valid splice segment that can be 

5 seamlessly spliced is an MPEG-compliant splice segment. 
An MPEG Splice Segment (SS) is defined at the transport 
level and includes functionality at the video (and audio) 
levels. An information-bearing splice segment may be as 
short as a single frame. A splice segment may even be a zero 

10 frame length segment (although such a SS might be MPEG 
non -compliant). Such a zero-length segment is simply an 
in-point followed by an out -point (i.e., an "in-out -point"). A 
SS may be also be very long, including many GOPs. In 
general the length of a SS is not constrained and the SS 

15 should include multiple out-points to enable seamless exit- 
ing from the segment. A possible exception is a SS com- 
prising a television commercial. The television commercial 
SS can be deliberately produced without defined out-points 
so that exiting the commercial segment is not seamless. 

20 An MPEG SS should be an NPEG compliant stream 
having consistent transport stream and elementary stream 
time stamps (e.g., PCR, PTS and DTS) and an associated 
delay parameter (e.g., a VBV delay), thereby allowing a 
decoder to properly decode and present the information in 

25 the SS. The first information frame (e.g., video access unit) 
at an in-point of an MPEG video SS must be an I -Frame. The 
second frame shall not reference information frames prior to 
the in-point (i.e., if the second frame is a B -frame, the 
B-frame may not reference frames prior to the in-point). The 

30 last frame before an out-point should not be a B-fxame (in 
display order). An audio SS will have an in-point consisting 
of the beginning of an audio frame and an out-point con- 
sisting of the last byte of an audio frame. There may be other 
constraints placed on the stream to address issues of, e.g., 

35 coding error-build-up, tuning-time and minimum picture 
quality. 

The in-point of a video SS must begin with a sequence 
header, although the SS may contain multiple sequence 
headers. ASS may contain additional header information to 

40 indicate that the sequence header is also an in-point. It is 
necessary to distinguish the SS in-point sequence header 
from a sequence header included for tuning-time or picture 
quality, since seamless splicing can only be guaranteed on 
in-points. Since the ia-point should follow an sequence end 

45 code (SEC) code it is desirable to include the SEC code just 
before the in-point, thereby obviating the need to include the 
SEC on the end of an out-point. The out-point may include 
the SEC. An MPEG-type splice count-down, if used, must 
end (i.e., equal zero) at the out -point. 

50 FIG. 1 shows a block diagram of a compressed bitstream 
splicing system 100 including the invention. The system 100 
includes a first compressed bitstream stream source 110, a 
second compressed bitstream stream source 120, a splicer 
300, a controller 105 and an optional splice monitor 130. 

55 The first compressed bitstream stream source 110, illustra- 
tively a "live feed" from a transport stream encoder, pro- 
duces a first MPEG-compliant transport stream S6. The 
second compressed bitstream stream source 120, illustra- 
tively a server (e.g., a video disk, tape machine, or other 

60 storage device) which stores video and audio elementary 
streams and transport encodes the stored streams to produce 
a second MPEG-compliant transport stream S7. The stored 
information may comprise, e.g., advertisement or local 
programming information to be spliced into the first trans- 

65 port stream. The splicer 300 selectively couples one of the 
two input transport streams S6, S7 to a transmitter or other 
subsystem as an output stream S9. An optional splice 
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monitor 130 monitors various parameters of the spliced the timing relationships between desired operation and 

output signal S9, e.g., delay parameter, buffer utilization actual operation, continuous-flow streams, server-generated 

information, synchronization, bitstream source and the like. streams and remotely-generated streams. 

The optional splice monitor 130 is responsive to the con- The timing relationships between desired operation and 

troller 105 and the splicer 300. 5 actual operation will be discussed first. At some operational 

The splicer 300 receives the first transport stream S6, unit, e.g. a Play-to-Air Switcher, a decision to switch 
illustratively a television program produced by a first source, streams must be made. The source of an output stream is 
and the second transport stream S7, illustratively an adver- actually switched in response to that decision, 
tisement produced by a second source. In response to a The decision to splice may be content related, such as a 
control signal SELECT, the splicer produces an output 10 switch from a from-stream to a to-stream when a content- 
signal S9 comprising either the first S6 or second S7 related data element is encountered in one of the streams, 
transport stream. The control signal SELECT may include For example, the from-stream may be monitored and, in 
priority information which causes the splicer 300 to respond response to the detection of, e.g., a black-screen or a scene 
immediately, within a defined time interval or when certain change, a splice decision may be made. This operational 
conditions exist (i.e., specific alignments of stream entrance is decision does not require synchronization. Rather, the deci- 
or exit points). The splicer 300 produces a signal sion requires that the splicer (or a controller) analyze, e.g., 
ACKNOWLEDGE which is used to acknowledge the the from-stream to detect the data element. The decision to 
SELECT signal and provide specific details about the splice splice may also be data-flow related, such as a switch from 
operation (e.g., exact time of splice, error conditions and the a from-stream to a to-stream on some particular packet or 
like). The operation of the splicer 300 is described more 20 upon the start or stop of information flow, 
fully below with respect to FIG. 3. -j^ decision to splice may be time-related, such as a 

The actual splicing operation is the process that takes switch from a program to commercial at noon. Time-related 
place within the splicer 300 that does what is necessary to decisions must be referenced to the splicer's local frame- 
actually switch amongst the bitstreams. This involves of- reference. A message-passing process passes the decision 
stopping, in an orderly manner, the flow of packets from the 25 information to the splicer in time for the splicer to be ready 
from-stream; starting, in an orderly manner, the flow of to make the splice in its frame-of-reference. Given that the 
packets from the to-stream; and adjusting the header infor- decision to splice at some time has been made, the splice will 
mation in the output stream. During some interval, packets be made at the next available splice point, based upon the 
from both the from-stream and the to-stream are likely to be from-stream and the to-stream. 

intermixed. Th e decision to splice may be may be event driven, such 

Splicing operations must be synchronized to be seamless. as the pushing of a button (e.g., the director's "take" 

To ensure that input streams arrive at the appropriate splicers command, as depicted in the splicer 100 of FIG. 1). When 

at the time they are needed several synchronizing operations the message indicating the event arrives at the splicer, the 

may be performed. It is assumed that the output stream is 35 actions are the same as those for a time-related decision 

continuous and that the actual splice is taken to be a change whose time has arrived, 

in the content of the output stream from a from-stream to a Some form of acknowledge message may be required, 

to-stream. The time stamps in the output stream should also ^ message, when delivered to the originator of the splice 

maintain continuity from one stamp to the next (this is decision (e.g., the controller), will allow an intelligent 

related to stream content) and the splicing mechanism 4Q cno ice to be made about time-outs, and actions like panic 

should adjust the output stream time-stamps. In the absence non-seamless splices. Time-outs and determinations about 

of time- stamp continuity in an MPEG system, the MPEG corrective actions to remedy splice failures is a policy matter 

"discontinuity" header flag should utilized such that an for the originator of the splice decision. Time-out and forced 

indication of a new time stamps (or time stamp switch may be a service implemented by the splicer but only 

discontinuity) is provided to a decoder. 4S as a convenience. 

To accomplish the adjustment the splicing process must ^ operational unit (e.g., splicer or switcher) may feed 
have some notion of time, since this local notion of time that back an appropriate acknowledgment message to a control- 
must be used to produce the output time-stamps. The splic- ling cntity> contents of sucn a feedback message may 
ing process gets its notion of time from some timing source inchlde one or more of the following parameters: 1) a splice 
such as the OC-12c interface and the current time is derived 50 did or d id not take place; 2) the local time-of-day that the 
from either stream content or set-time messages. The local sp ii ce occurred; 3) the delay-parameter value of the 
notion of time must be moderately continuous and well to-stream; 4) the delay-parameter value of the from-stream; 
behaved. When splicing, both the end of the from-stream 5 ) the current (post-splice) sync-buffer delay (e.g., in delay 
and the beginning of the to-stream must be available at the seconds); 6) the future time a splice will take place (if the 
actual splice hardware that is producing the output. In 55 switcher can compute this value); and 7) any exceptions or 
addition, all buffering within the splicing process must be errors. Exceptions and errors may include the fact that no 
finite and defined. splice took place, that the decision parameters passed by the 

In addition to the above issues, there are synchronization controller were incorrect (e.g., syntactically or logically), 

issues to be considered. For example, it is important to that the to-stream was not ready, that a time-out occurred or 

consider the effect of packet jitter on the splicing process. If 60 that an audio-failure occurred (e.g., the dropping of an 

any additional information is required, beyond that con- excessive number of audio frames), 

tained within the actual streams being spliced (e.g. priority Additional information that may be of value includes: 1) 

information, source identification, error codes and the like), the amount of time that the audio information from the 

the additional information must be properly synchronized from-stream will be needed; 2) an indication that the inputs 

with the actual splice streams. 65 are buffered correctly and ready for a new splice; and 3) 

There are several conditions that are of interest with other information useful to the controller or the splicing 

respect to synchronization of the splicing function. These are process itself. 
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The precise time at which a seamless splice takes place accumulates 2.6 seconds. A six second buffer, initialized to 

may not be pre-determined, since the seamless splice a 3 second fill, is adequate to absorb clock drift, 

depends upon the arrival of an in-point in the to-slream. In Splia . mom toring is an important aspect of splicing, 

the case of a decision to splice seamlessly there are several esp ecially in a studio environment. Content-related moni- 

sub-decisions which must be made about what to do if the 5 |Q • comprise tne steps of viewing an image on a 

splice does not take place within some time limit. The d - , device (; « moniton responsive l y changing 

choices are as follows First, simply wait for a seamless ^ ters of the bitstr6ams producin g the image (e.g., 

sphce to occur. Depending upon studio operational goals » y Optional splice monitor 130 may be used for 

tins may not be acceptable. Second define a fixed time-out monitorjQg by> e . g ., a director. Another form 

period and, if the splicer has not spliced w.th.n the defined „ f monitorin is the qual Ttative assessment of a monitored 

time-out penod, perform a non-seamless sp.Ke (.£ switch bime&m ^ ^ monftor 130 fae ^ tQ 

streams in las controllable a .manner as .possible) -WhiUtbs re(rieve li(ative information from lhe sp i iced outpu , 

is a simple approach, it limits decision flexibility Third, * parameter, buffer utilization 

simply wait for the splice to occur but acknowledge th.s Nation, sy ° ch ronization information, biKtrea m source 

cnditaon to the controlling entity. Whatever process gener- „ identiflcation * nd me lflce ^ tional lic6 monitor 130 

ated the sphce decision may then decide to allow a non- . . 1t in , . iU r ann . 

^. . «, i . i a t. -i is responsive to the controller 105 and the splicer 300 to 

seamless sphce. This allows reasonable operational flexib.l- ^ ^ ^ information ^ return> ln 

ity Fourth, provide a programmable tune-out, optionally ^ £ um Qr tQ , e ^ mative information 

including a default time-out and optionally including an directl ^ ^ controlIcr 105 and the licer 300 for further 

acknowledgment or this condition to the controlling entity. 2 o 

This option allows the controlling entity to alter the time-out v 6 ' . . 

value, thereby providing the greatest operational flexibility. Wlth KSP"* t0 content-based monitoring, a director (i.e., 

It should be noted that a time-out is a secondary operational human ) ma y be the vanous bitstreams via monitors 

feature and, therefore, an exception to normal operation. (*•••• decode,s drivin g d *P^ and responsively sphcing to 

The synchronization of continuous-flow stream splicing 25 P codnc * an . 0U <P ut , stream » P ro S ram >- Th f. ,ssl,e ,' s 

will now be discussed. The streams arrive and must be especially significant m production or live switching with 

buffered (e.g., one second's worth of information) within the cpmptox-GOP streams^ The problems are greatly minimized 

splicer in some manner so that, e.g., to-stream in-points are lf °f rat ' n S " a 'ow-delay (e.g., -frame only) production 

_ , , .« . „„- ir i n iu» frnm C f ro ; m if tu a mode. When the director indicates that a splice is to be made 

available when out-points occur in the trom-stream. It the „ „ , ... . 1M , . , , . „ iU 

_ . u rr . . r . „ « . „ + , n t . „ a now, that decision is likely to be based upon what the 

amount buffered is insufficient (e.g., more than a second in ' , * * f • , 

elapses between successive in-points in a from-stream), then director sees . and •f' 8 t0 **r M r a decoder ™ herentl y 

the buffer will overflow and will contain invalid information. P resents ,,s P^ture after some delay from its input bnstream 

«... . , , . 4 , f (e.g., part or all of end-to-end delay and some additional 

This condition is remedied by an appropriate number of \ & v * ^ , , ^ . i e <• 

in-points and out-points being inserted into the bitstreams. If decode and £ ™ ^ elay) ' the modes o£ ^Tl^I* the 

, , r . u . - . , . . t ft director in an MPEG studio environment will be dinerent 

bitstreams do not have in-points and out-points often 35 . t ^ r « 

enough, then those bitstreams can not be seamlessly spliced ^ current NTfC-related operations. For purposes of the 

at those times. Moreover, to the extent that there is packet or Allowing examples, assume that a director is operating m an 

cell jitter in the arrival time of input bitstreams, a first-in, MPEG studio environment, that sphce segments are % 

first-out (FIFO) buffer (with output clocked at nominal data second and ^ hat the ^-to-end delay of the spliced 

rate) is expected to smooth the flow. 40 stream 15 Yl second - 

-Hie synchronization of server generated streams will now A first exam P le 1S 3 "soonest" mode of operation. In this 

be discussed. Server-generated streams must be carefully mode > the director P resses a " lake " button TAKE based upon 

generated so that the data does not arrive at the splicer too an event seen on an out P ut monitor 132, a from-stream 

early or too late. If the data arrives too early, there is some moriitor 136 or a to-stream monitor 134. A queued up (e.g., 

risk of overflow of an input buffer. If it is assumed that the 45 server-stored) to-stream is ready and aligned at an in-point. 

splicer has enough synchronization buffering to hold a Within 1/4 second of the s P hce command an out-point will 

second or so of video, then it would seem that server streams arnvc at cnd of me from-stream sync-buffer and the splice 

can be delivered in any pattern of flow that never exceeds the wil1 be made - h must be noted lha ! the from-stream contains 

just-in-time limit, and the one-seconds arly limit. Of course, U P t0 1/4 of dela y* 0ne momtor dela y later ( % xcond ) 

there may be peak rate limitations on the splicer. 50 thc sccnc on thc output momtor chan S cs - 

The synchronization of remotely-generated streams will If the director responded to a scene on the output monitor 

now be discussed. It must be noted that any stream pro- 132 > lhe amount of output monitor delay (i.e., the time 

cessed in a studio containing the splicer is expected to have between the "take" command TAKE and a change in scene 

the same reference clock rate. Remotely-generated streams, 00 the 0Ut P ut monitor 132) is between and one second. If 

by the time they have reached a splicer, should be the same 55 tDe director responded to a scene on the from-stream moni- 

as locally-generated real-time streams. To reference tor 136, the amount of from-stream monitor delay is between 

remotely-generated streams to a local master clock the ^ and ^ second and the output monitor delay is second, 

remote source may be genlocked to the local studio. This can If thc director responded to a scene on the to-stream monitor 

be done via a reverse channel or by locking both to an 134, the to-stream monitor 134 is continuous (i.e., no 

external reference, such as a timing signal derived form the 60 monitor delay) and the output monitor delay is negative Va 

Global Positioning System (GPS). If there are two indepen- seconds (i.e., the scene changes Va second after the "take" 

dent studios, each with an independent master clock, and ouUon TAKE is pressed and the image displayed occurred Ya 

each is doing a remote feed to the other, then one will be second prior to the press of the button), 

delivering data too slow for the other, and one will be A second example is the "next" mode of operation. In this 

delivering data too fast. Another method is to delay a remote 65 mode, a queued up to-stream is flushed from a to-stream 

feed by a time equal to the maximum clock drift over some synchronization buffer and the next segment beginning with 

operating interval. A 30 ppm. drift rate, over 24 hours, an in-point is queued up within up to Va second. The 
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to-stream synchronization buffer also has zero to Ya seconds 
of random delay. When the in-point arrives the splice is 
made. 

If the director responded to a scene on the output monitor 
132, the amount of output monitor delay is between X A and 
one second. If the director responded to a scene on the 
from -stream monitor 136, the amount of from-stream moni- 
tor delay is between Vi and Y* second and the output monitor 
delay is second. If the director responded to a scene on the 
to-stream monitor 134, the to-stream monitor 134 is con- 
tinuous and the output monitor 132 switches to a new scene 
between zero and V* second later. 

The choice of "soonest" or "next" mode of splicing is an 
operational one, and may be based upon which disconcert- 
ing effect (delay or back-up) is least objectionable. To 
alleviate these effects an amount of delay may be inserted 
into the splicer inputs. If this delay matches the monitor 
delay, and the monitors are connected to the inputs of the 
delays, then the apparent delay between monitor scenes and 
button action is less, but the delay to final output is greater. 
In addition, a separate monitor control unit may be built to 
simulate the bit-stream switching and show the simulated 
results of the bitstream switch, thereby providing more 
flexibility to the director. 

FIG, 3. shows a detailed block diagram of the splicer 300 
of FIG. 1. The splicer 300 selects one of a first input 
bitstream S6 and a second input bitstream S7 as an output 
bitstream S8. The output bitstream S8 is optionally time 
stamped to produce a retimed output stream S9. The first and 
second input bitstreams S6, S7 are, illustratively, MPEG- 
compliant transport streams including at least video and 
audio elementary streams. The video and audio elementary 
streams may be in a packetized elementary stream (PES) 
format. 

For purposes of the following discussion it is assumed 
that the second bitstream S7 is currently selected as the 
output bitstream (i.e., S7 is the from-stream) and the first 
bitstream S6 will be selected as the output bitstream (i.e., S6 
is the to-stream) after a splicing operation. 

The first input bitstream S6 is coupled to a first bitstream 
examiner 310A and a first synchronization buffer 320A. The 
first bitstream examiner 310A examines the first bitstream 
for entrance points which have been included in the first 
input bitstream S6. When an in-point is found, the contents 
of the synchronization buffer are discarded (i.e., the buffer is 
"flushed") and the in-point is stored in the first memory 
portion of the synchronization buffer. The synchronization 
buffer may be constructed as a first-in, first-out (FIFO) 
buffer. The process of searching for in-points and flushing 
the buffer is repeated until the first input bitstream S6 is 
selected by the splicer. In this manner, there is always an 
in-point positioned at the end of the synchronization buffer 
of the stream that is not being currently output. The output 
bitstream S3A of the first synchronization buffer 320A is 
coupled to a switch controller 340 and a first working buffer 
330A. The first working buffer 330A produces an output 
signal S4A which is coupled to a packet switching unit 350. 

The second input bitstream S7 is coupled to a second 
bitstream examiner 310B and a second synchronization 
buffer 320B. If the second bitstream were not presently 
selected as the output stream, then the second bitstream 
examiner 310B and synchronization buffer 320B would 
operate in the same manner as described above with respect 
to the first bitstream examiner 310A and synchronization 
buffer 320 A. The second bitstream examiner 3 10A examines 
the second bitstream for exit points which have been 
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included in the second input bitstream S6, In the "selected 
mode" of operation, the second bitstream examiner 310B is 
not used and the second synchronization buffer 320B serves 
as a constant delay buffer which produces a delayed bit- 

5 stream S3B. 

The delayed bitstream S3B is coupled to a working buffer 
330B and a switch controller 340. The second working 
buffer 330B produces an output signal S4B which is coupled 
to packet switching unit 350. The second working buffer 

10 330B holds the selected bitstream long enough to allow for 
overlap of old audio packets with current video packets. This 
allows audio frames to continue to completion after a splice 
is made. The synchronization of audio and video frames are 
discussed in more detail below and in U.S. patent applica- 

^5 tion Ser. No. 08/864,321, filed May 28, 1997 and incorpo- 
rated herein by reference. 

A splice decision is made by a controller (e.g., controller 
105) and coupled to the switch controller 340 via a control 
signal SELECT. Assuming that the splice decision equates to 

'° the command "splice seamlessly at the next opportunity," 
the switch controller 340 responds by scanning the currently 
selected output stream (i.e., bitstream S3B) for out-points. It 
is assumed that an in-point is positioned at the end of the first 
synchronization buffer 320A. When an out-point arrives on 

15 the from-stream, the switch controller 340 causes, via a 
control signal A/B, the switch 350 to begin coupling video 
packets from the to-stream through the switch to an optional 
header adjuster. At an appropriate time any audio packets 
within the to-stream are also switched. 

The optional header adjuster 360 alters time-stamps in the 
selected output stream S8 to produce a retimed output 
stream S9. The retiming of the program clock reference 
(PCR), presentation time stamps (PTS) and decode time 

5 stamps (DTS) of the selected stream S8 may be necessary to 
ensure that the splice is, in fact, seamless to a decoder. The 
header adjuster 360 includes a 27 MHz (local) station clock 
362 which is utilized by a local PCR and PCRB generator 
364. To retime the presentation and decode time stamps it is 

0 necessary to partially decode (i.e., packetized elementary 
stream (PES) layer) the selected transport stream S8. The 
partial decoding and retiming of the PTS and DTS is 
performed by a PTS and DTS detection and retiming unit 
366 to produce a PTS and DTS retimed stream S8R The PTS 

5 and DTS retimed stream is transport encoded and time 
stamped by PCR detection and retiming unit 368 to produce 
a retimed transport stream S9. An alternate embodiment of 
a header adjuster is the PTS-DTS retimer discussed in more 
detail in U.S. patent application Ser. No. 08/864,326, filed 

Q May 28, 1997 and incorporated herein by reference. 

As previously mentioned, the invention may be imple- 
mented using a general purpose computer system that is 
programmed to perform the various functions. The embodi- 
ment of FIG. 3 may be implemented as a computer program 

5 utilizing portions of memory to provide buffering, and an 
algorithm directed to the examination, control, switching 
and header adjustment functions. As previously mentioned, 
the splicer 300 produces a signal ACKNOWLEDGE which 
is used to acknowledge the SELECT signal and provide 

3 specific details about the splice operation (e.g., exact time of 
splice, error conditions and the like). A routine for splicing 
will now be described with respect to FIG. 2. 

FIG. 2 illustrates a splicing routine in accordance with the 
invention. The splicing routine is entered at step 202 when 

5 the decision to splice is made. For the purpose of this 
discussion, it is assumed that the decision is to seamlessly 
splice from the currently selected (from) stream S4B to 
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another (to-stream) stream S4 A. The decision is examined at Each of the islands receives a plurality of information 
step 204. If the decision of step 202 is to splice as soon as streams. For example, islands 401 and 402 each receive 
possible, then the routine proceeds to step 208. If the information streams from a NETWORK FEED and a 
decision is to splice at the next in-point (e.g., skip the LOCAL FEED. Controller 105 communicates with each of 
presently buffered GOP in the to-stream), then the synchro- 5 the islands via a control channel C. The control channel C is 
nization buffer (e.g., 320A) is flushed. When the to-stream used to direct the flow of information throughout the studio 
synchronization buffer (e.g., 320A) has a valid in-point (i.e., between islands) and to direct the processing of the 
stored (step 208) and the from-stream is at a valid out-point information within the islands. The controller 105 provides 
(step 210), the splice is made (step 220) and the routine is the splicing decisions and any necessary parameters asso- 
exited (step 230). The context in which a splicing decision 1Q dated with the intended splice. The islands respond by 
(step 202) is made is relevant to the amount of information performing, monitoring and acknowledging (via control 
necessary to perform a seamless splice. If the splice decision channel C) the various splicing operations, 
is made in the context of building play-to-aii -edit lists, it is A d ^ ul studio accordi l0 lhe invention may be 
necessary for the streams to be spliced to have the same ^ ^ imerconnected of „ lici ^Und*" 
value of delay-parameter. If the splice decision is made in , , • c j 
the context of creating a live production, it is necessary for 15 which perform particular processing tactions on received 
the streams being spliced to have matching delay-parameters bitstreams to produce output bitstreams. This is because the 
and splice points which occur often enough to meet opera- * lands form individually distinct operating environments 
tional requirements. If the splice decision is made in the ( e S > stora S e environments, editing environments, process- 
context of creating a live production, and the production in g environments and the like) which cooperate with each 
involves information stored on servers, it is helpful to know 20 otner via a controller 105 to produce one or more output 
that anticipated splice-points are about to arrive soon (via, bitstreams (e.g., S9, OUTPUT STREAM). Each island oper- 
e.g., splice point countdowns or splicing tables). The splic- ates at a known delay-parameter value and all splicing 
ing decision and related matters will be discussed in detail within an island is (ideally) seamless. The splicing and 
below. processing functions are under the general control of con- 
The invention will now be described within the context of 2s Roller 105, but may be locally controlled as necessary. For 
a digital television studio including a number of distinct example, an operator sitting at an editing station may 
operating environments (such as servers or edit-suites) logically comprise one island. The streams to be edited are 
which receive, process and transmit various information routed to an editing island (e.g., island 407) in response to 
streams. The operating environments, or "islands of commands transmitted via control channel C from controller 
interoperability," may be interconnected to perform one or 3D 105. In response to the commands a signal may be switched 
more operations on the various information streams. The through several islands (e.g., 406 and 300) prior to being 
studio output may be delivered to end-users (e.g., the public) stored in a storage unit (not shown) at the editing island (e.g., 
via ATSC broadcast, cable, telephone and satellite transmis- 407). 

sion and the like. The studio output may also be stored for An alternate mode of studio operation is to controllably 

later use in, e.g., a server or on CD-ROM or video tape. The 35 operate one or more islands in a non-seamless mode. The 

invention is also useful in video teleconferencing and other non-seamless mode may be required in several circum- 

applications. stances where a splice or other transition between bitstreams 

While the streams delivered to broadcast customers must must occur rapidly, and a range of bitstream degradation is 

meet, e.g., ATSC standards, it is not necessary to deliver all permissible. It must be noted that non-seamless switching 

of the internal studio information. For example, high bit-rate 40 may produce errors which are propagated to subsequent 

studio formats are useful only within the studio or studio- islands receiving a degraded bitstream. These errors maybe 

like environments. When dealing with splicing, there may be mitigated, if necessary, by, e.g., dropping damaged or infe- 

information within the stream that is meaningless to con- rior access units or groups of access units (e.g., video 

sumer decoders, but which is necessary to studio splicing. frames) or by adding additional access units. For example, 

FIG, 4 depicts a block diagram of digital studio compris- 45 if a to-stream having a short delay-parameter is to be spliced 
ing a plurality of interoperable islands and including the onto a from-stream with a long delay-parameter, the splicing 
invention. The digital studio 400 of FIG. 4 includes in terop- operation is unlikely to be seamless (i.e., the buffer will 
erable islands 401, 402 and 404-409. The digital studio 400 likely overflow). In this case, frames may be dropped to 
also includes a first compressed bitstream stream source 110, avoid the overflow condition. Also, when a to-stream with a 
a second compressed bitstream stream source 120, a splicer 50 long delay-parameter is to be spliced onto a from-stream 
300, a controller 105 and an optional splice monitoring unit. with a short delay-parameter, the splicer needs to adjust time 
The first compressed bitstream stream source 110, illustra- stamps to cause a number of frame repeats (i.e., add frames) 
lively a "live feed" from a transport stream encoder, roduces while the buffer fills. The buffer may also be increased by 
a first MPEG-compliant transport stream S6. The second splicing short, all-black frames on the end of a short delay - 
compressed bitstream stream source 120, illustratively a 55 parameter sequence to build up the value of the delay- 
server (e.g., a video disk, tape machine, or other storage parameter in current use. 

device) which stores video and audio elementary streams In the exemplary embodiments of FIGS. 1, 3 and 4, 

and transport encodes the stores streams to produce a second splicing operations take place in operational units (e.g., 

NPEG -compliant transport stream S7. The first and second splicing islands), such as routing switchers, play-to -air 

compressed bitstream sources 110, 120 operate in substan- 60 switchers, production switchers or other switchers, 

tially the same manner as previously described with respect Therefore, it is desirable to support a plurality of data 

to the bitstream splicing system 100 of FIG. 1. The digital formats and bitrates. For example, the so-called 422@HIGH 

studio 400 includes a controller 105 which performs those and 420@HIGH television studio formats each support 

functions previously described with respect to the bitstream multiple picture formats and bit rates. Therefore, it may be 

splicing system 100 of FIG. 1, and other functions which 65 necessary to splice, e.g., a bitstream comprising a 1280 by 

will be described below. Island 300 roughly equates to the 960 picture element, 60 Hz Progressive Scan picture onto 

splicer 300 of the bitstream splicing system 100 of FIG. 1. the end of a bitstream comprising a 1920 by 1080 picture 
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element, 59.94 Hz interlaced picture. Moreover, it may also level, transport level and PES level. There are also oppor- 

be necessary to splice a 45 Mb/s stream onto the end of a 155 runities to insert markers in the elementary streams. 

Mb/s stream. Both in-points and out-point should be marked and, 

Both of the above example splices may be seamlessly ideally, the marking should occur at the system, transport 
made if the streams being spliced have matching delay 5 and PES levels. In addition to the insertion of in-point and 
parameters. Therefore, it is important that the controller that out-point markers, the delay-parameter associated with the 
makes the splice decision know the delay parameters of the stream or splicing segment and an audio offset (i.e., a 
various streams to be spliced. The delay parameter of a displacement of audio-frame boundaries from associated 
stream may be calculated by an operational unit receiving a video frames) should also be inserted into one or more layers 
stream or included within the stream as part of the stream's 10 of the information stream. The MPEG count-down feature 
header information. In the splicer 300 of FIG. 3, switch should also be used to indicate that, e.g., an out-point is 
controller 340 includes a bitstream calculator which calcu- approaching (decreasingly positive countdown) or an 
lates the delay parameters of the input streams S6, S7. Is in-point has been transmitted (increasingly negative- 
should be noted that the delay parameter calculation may countdown). For various business reasons it may be desir- 
also be performed by the bitstream examiners 310 A, 310B 15 able to remove these markers prior to transmitting an MPEG 
or the optional splice monitor 130. or ATSC signal to an end-user (i.e., consumer). While 

Another critical aspect of splicing information streams is end-users might want to splice the video, it is important that 

the determination of in-point and out-point locations in the lhe y not be able t0 cli P out commercials automatically, 

streams to be spliced. To properly perform a seamless splice The above-described redundancy of marking provides a 

it is necessary to find the in-point of the to-stream and the 20 maximum flexibility to system designers and provides a 

out-point of the from-stream. Moreover, a splice segment redundancy of operation which helps to ensure that splicing 

may include in-points and out-points having different delay- operations are, in fact, seamlessly made (i.e., made at 

parameter values. There are several options available for appropriate in-points and out-points). 

finding the appropriate splice points. „. ^ 

„. , . * , , , 25 Bitstream Generation 
First, the entire to-stream or from-stream may be analyzed 

by the splicer in real time (i.e., "on the fly"). A real-time To help ensure seamless splicing it may be necessary to 

analysis is difficult for a to-stream because an in-point create the bitstreams to be spliced in a certain manner. There 

cannot readily be deduced from the stream without playing are two facets to the creaction of bitstreams that can be 

the stream to its end. Even if it were known that the stream 3Q spliced; the creation of the stream content, and the insertion 

is well-behaved, the length of an I-frame is not known in °£ appropriate splice control information (i.e., in-point and 

advance. By the time the first I-frame has ended, and its out-point markers). For simplicity, assume that a desired 

length is known, it is probably too late for the information value of the delay-parameter is known in advance. In 

to be used. It must be noted that this problem may be addition, other goals, such as how often an in-point is 

overcome by using, e.g., a more powerful computing device, 35 wanted, are also known. 

A real-time analysis is easier for a from-stream because the For the simplest case, encoding all I-frame low-delay 

delay-parameter of the from-stream is known (from the format, the creation of the bitstream becomes a matter of 

in-point or otherwise), the presentation time -stamps in the rate-control. For each frame, there is a not-to-be -exceeded 

stream indicate when frames leave the decode buffer, and bit-count. The rate -control task is to encode each frame with 

bit-counts (or packet counts) indicate when the frames enter 4Q the best quality possible within the bit-budget. The per- 

the decode buffer. The frame rate is also known from frame bit budget is computed as the transmission-bit-rate 

sequence headers. With the foregoing information it divided by the frame-rate. 

becomes possible to calculate, at the start of a new frame, for complex GOP encoding a forward analysis of the 

that the preceding video has left the buffer in an out-point created stream may be made. The allocation of bits among 

status. This information arrives only a little bit late. 4S frames must be done to assure that a decoder buffer doesn't 

Second, an external table may be created to contain underflow. A first constraint which may be applied to the 

indications of where splice points are. This approach created stream is the defining of a splice segment as a fixed 

assumes that the information about in-point and out-point GOP structure (e.g., 13 frames arranged in the following 

locations has been computed elsewhere (e.g., during a display order: ". . . IBBPBBPBBPBBP . . . "). This approach 

stream encoding process). This approach requires that the 50 is straightforward a the expense of unnecessarily degraded 

in-points and out-points be indexed in some manner (e.g., picture quality. For example, a scene cut on the last P frame 

Nth packet from a marker, first packet after a time-of-day of an ". . . IBBPBBPBBPBBP ..." GOP would be 

reference, and the like). This approach also requires the reproduced with a very small bit budget. Unfortunately, 

updating of a splice table associated with an information there is no single GOP structure that is ideal for all appli- 

stream as the information stream is processed, and the 55 cations. Moreover, the loss of flexibility implied in this 

transmission or regeneration of the table as the information approach is probably unacceptable, 

stream is transmitted (e.g., via satellite link). It should be A second constraint which may be applied to complex 

noted that the use of separate information tables to deter- GOP encoding is the insertion of in-points at out-points at 

mine splice locations is practical within a server or island of predetermined time intervals (e.g., 2 and 0.5 seconds, 

operability, though this use of tables may be less practical 60 respectively). This approach does not require the use of a 

when transmission encoding and decoding are to be per- specific GOP structure, therefore the encoder is free to select 

formed on the stream. frame type based upon the input pictures. 

Third, in-point and out -point markers may be placed There are various rate-control issues to be resolved when 

within the information stream directly. An MPEG compliant switching between MPEG streams or splice segments. As 

information stream includes header portions where such a 65 such, it may be necessary to constrain the contents or size of, 

marker may be included. There are header portions suitable e.g., coded picture bitstreams. One rate-control issue 

for in-point and out-point marker insertion at the system involves the amount of data transmitted to a decoder buffer. 
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For example, the decoder buffer will not overflow if the information stream, a time stamp (e.g., PTS or DTS) in a 

buffer contents (measured in bits) at any out-point is less- video or audio stream, a reference time (e.g., PCR) in an 

than-or-equal-to the decoder buffer contents (measured in information stream or some other detectable event (e.g., an 

bits) measured at the most recent in-point. It is not necessary input stream PID changes). The logical events may be 

to know the actual number of bits, it is only necessary to 5 combined in a logical manner to determine a splice time and 

ensure that the number of bits in the decoder buffer does not select appropriate streams for splicing, 

increase from in-point to out-point. It must be noted that After a decision is made it is communicated to the 

so-called "stuffing bits" are not counted, since these bits are appropriate operational unit. Since there is a possibility of 

only included to meet specified transmission rates and crrors ( e g ^ due t0 t h e complexity of combining multiple 

disappear from the buffer when the real bits are used (i.e., id logical events) it is important for the controlling entity to 

stuffing bits do not accumulate in the buffer). monitor the splicing process closely and adapt to error 

Another rate control issue involves the presentation time conditions and other conditions, 

of decoded information frames in relation to the time the As a result of the decision, and part of making the splice 

buffer receives the next frame. For example, the decoder ^ the process of assuring that the correct bitstream is 

buffer will not underflow if, at an out-point, the time *5 t0 me correct i nput 0 f the operational unit at the 

(measured in time units) to display the not-yet displayed correct time. This is a function of overall system operation, 

buffer contents is greater than the time (measured in time and ma y involve synchronization of information flow from 

units) for the first I-frame to be delivered to the decoder at a server to tnc sw it c her. After receiving the splicing decision 

the specified bit rate. In other words, the next I-frame (the tne splicer will make the splice. The process of seamless 

first frame of the to-stream) must be delivered to the decoder 20 sp ij cmg rc q U i re s that the to-stream is entered at an in-point, 

buffer before the buffer is emptied. tne from-stream is exited at an out-point and, possibly, the 

The above-described amount of time may be defined at management of one or more buffer and timing parameters, 
the " Delay- Parameter" for the stream. The frame sizes 

(measured in time to transmit the frames at the operating bit Illustrative Examples of Video Spb'cing 
rate) must be consistent with the operating delay parameters 25 ^ above-described concepts will now be illustrated 
to ensure seamless splicing. The delay parameters are the using several examples of MPEG bitstream splicing opera- 
end-to-end VBV size (measured in time) and the VBV tions ^ first example ^ an & Nframe low delay splicing 
contents (measured in time) at the beginnmg/end of a example. A to-stream comprises either 24 or 30 frames per 
stream. An additional, globally defined value is the maxi- second (fps) vide0 streams including only i_f rames . The 
mum size of physical buffers (in bits). This maximum size dday parameter of me to-stream is equal to one frame time 
must be greater than the maximum VBV size implied by the at the slowest frame rate ^ 42 mS tf 2 4 fps). In this 
MPEG profile and level indication criteria. Finally, the example each of the I-frames contains fewer bits than can be 
decode time stamp at an out-point of a from-stream should sent in one ^ of d is play ^ ^ 0De 42 m s frame time) 
be one frame time of the stream greater that the DTS and at thc bit ratc for the framc [f ^ bit fate ^ 15fJ a 30 
PTS of the last frame of the from-stream. fps frame nutans no more than 5 Mb. If the bit rate is 150 

As previously mentioned, it is important to distinguish Mb/s, a 24 fps frame contains no more than 6.25 Mb. 

between a splicing decision and the actual process of splic- when the last bit of a f rom . stre am having the same 

ing. A splicing decision is made by some human. The parameters has entered a decoder buffer, the presentation 

decision may be made in the process of generating a list of 4Q time stamp indicating when the last frame is to be presented 

programming to be transmitted by a television studio or in has a value 42 mS in the future. Thus, if the from-stream 

real time as the studio is transmitting. The splicing decision wcre at 30 Hz (33 mS frame rate)j 9 ms . afler the out . po int 

may be made by some surrogate process, such as a prepro- the last frame of the from strcam will be taken from tQe 

grammed command to splice a station identification decoder buffer, and 33 mS later the first frame of the 

announcement into the studio transmission every day at 4S t0 . strcam wi n be needed. If the to-stream is also 30 Hz, the 

12:05 AM. The decision may be to splice at a particular time first frame will have been de livered 9 mS before it is needed, 

in the future or immediately. If me f r0 m-stream were at 24 Hz, and the to-stream were 

Several parameters of the streams to be spliced may be also at 24 Hz, the to-stream frame arrives just in time. If the 

known at the time of the decision, though these parameters from-stream were at 60 Hz, (17 mS) then when the last bit 

may change prior to the actual splicing operation. For 50 of the from-stream is delivered, the decoder buffer contains 

example, the splicing decision is usually made with some 2 frames (33 mS) and the decoder will not use the first of 

knowledge of the to-stream, such as the stream length, VBV them for 9 mS. If the bit-rates of the from and to sequences 

delay parameter and the like. It is possible for the from- differ, as long as the bits are delivered at the rate corre- 

stream to be unknown at the time of the decision (e.g., the spending to the sequence bit-rate, the time to load frames 

daily message is inserted into whichever stream is being 55 remains correct. To summarize, streams are coded with a 

transmitted at the time of insertion). bit-count between the in-point and following out-points that 

A decision contains the following elements. First, the is calculated from the bit-rate and the frame-time (i.e., 

operational unit which sources the to-stream, the operational bit-rate * frame -time). The presentation time -stamps are set 

unit which performs the splice and the stream or segments to values that all agree with the delay-parameter (i.e., first 

to be spliced. Second, the time the splice is to take place. The 60 frame presented delay-parameter after the first bit arrives.), 

time may be "now," a particular time of day or the occur- The second example is a complex GOP transmission 

rence of some logical condition. "Now" means make the format. For purposes of the second example, it is assumed 

next splice after the arrival of the splice -now message. The that stream is a 30 frame per second video (and associated 

now decision may arise from direct human action (e.g., audio) stream having delay-parameter of 250 mS, a display 

button-press) or some external controlling process deciding 65 order of". . . IBBPBBPBBPBBPBBP ..." and transmission 

to send a splice-now message. The logical condition may be order of . . IPBBPBBPBBPBBPBB ..." (where "I" 

the occurrence of a time-code (e.g., SMPTE) in a particular represents an I-frame, "P" represents a P-frame and "B" 
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represents a B-frame). This GOP structure includes in-points 
on the I -frames and out-points on the frame immediately 
preceding the I-frames. At each out-point, the PTS associ- 
ated with the last P frame is 250 mS in the future. The rate 
control ensures that the decoder buffer doesn't underflow on 
the I-frame. It must contain less than 250 mS worth of bits. 
The valid-MPEG constraint implies that the P-frame fol- 
lowing the I frame also doesn't underflow. It is not necessary 
that the I-frame use all 250 mS. If the I-frame does use all 
250 ms, the next P-frame must use less than 33 mS (i.e., if 
the I-frame used 230 ms, the next P-frame would be 
permitted, though not required, to use 53 mS). 

The third example is a multiple output example. For 
purposes of the third example it is assumed that stream is a 
30 frame per second video (and associated audio) stream 

having the following GOP structure: " IPPPPPPPPPP- 

PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPIPPPP ..." 
The stream is also assumed to have a delay-parameter of 250 
mS and a transmission bitrate of 20 Mb/s (i.e., 670 Kb per 
frame). If an I-frame takes 231 mS and P-frame takes 20 mS 
then, after 15 frames, the decoder buffer contents have 
subsided to a level below the level at the in-point to the 
stream. This may be calculated using an equation such as the 
following: 

(231+2(W)=33*(tf+l) 

The above equation represents the time for bits entering, 
starting with an I-frame 's 231 mS worth of data, followed by 
N 20 mS P-frames, being equated to the time for the same 
N+l frames-worth of bits to be taken from the buffer (N+l) 30 
frames, each taking 33 mS. In this example each I-frame 
may be an in-point and all the P-frames after the fifteenth 
P-frame may be out-points. 

The decoder buffer reacts as follows. At the splice point, 
the buffer contains 7 frames and a presentation time stamp 35 
indicates that the last frame is to be presented in 250 mS. 
The 7 frames each use 20/33 of the bits/frame quota (i.e., 
each P-frame frame is 400 Kb and the buffer contains 2.8 
Mb). Over the next 7 frames, the I-frame comes in, increas- 
ing the buffer contents. Since each I-frame adds 670 Kb and 40 
each P-frame taken out removes 400 Kb the buffer contains 
4.7 Mb after the seven P-frames are taken out. The I-frame 
is then presented, removing 4.6 Mb from the buffer and, 
therefore, leaving 100 Kb in the buffer. The delay in the 
buffer is approximately zero. Each P-frame now adds 400 45 
Kb in 20 mS and every 33 ms 400 Kb is used. Thus, delay 
in buffer increases by 13 mS every frame time. After 15 
frames, the delay stored in the buffer has reached the 
delay-parameter value. At this time a splice to another 
sequence may be made because the buffer is able to receive 50 
an I-frame. 

Compressed Audio Splicing 

The following discussion of splicing of compressed audio 
is limited to the issue of splicing combined video-audio 55 
streams in the audio-follows-video mode. The composition 
of streams from separately edited audio and video streams is 
not considered here. 

Compressed audio is carried in frames. Each audio frame 
is of fixed duration and contains a fixed number of bits. 60 
Unfortunately, the audio frame size, or duration, is different 
from any of the video frame sizes, or durations. This means 
that audio frames will not align with splice points. Audio 
frames can be considered to be randomly aligned with the 
video. Therefore, when making a splice, the alignment of the 65 
audio with the video will be different for the to -stream and 
the from-stream. 



It is desirable to ensure alignment of audio information to 
corresponding video information (i.e., "hp -sync"). That is, 
the audio and video must remain properly phased with 
respect to each other. A Presentation Time Stamp (P7S) 
exists in each audio stream. The audio and video PTSs refer 
to the same reference to allow the required synchronization. 
When the splice is complete, the to-stream becomes the 
output stream. It is important to note that, particularly due to 
audio constraints, the process of switching may extend in 
time before and after the actual switch instant. 

Audio information frames in an information stream are 
ideally located within a limited time difference from respec- 
tive video information arrival at the end of the decoder 
buffer. If there is a ¥2 second end-to-end video buffer delay, 
then audio packets should be approximately Vi second later 
in a transmission stream than corresponding (i.e., having the 
same presentation time stamps) video packets. If this 
assumption is correct, then the switching operational unit 
must save audio information from the from stream for this 
V2 second after the video switch. The source stream must 
continue for x h second after the splicer has switched to 
another stream. It is also amusing to contemplate rapid 
switching among several streams. 

Three approaches to splicing audio will now be described. 
First, the overlapped audio packets may simply be broken. 
This is not the most desirable approach because it relies 
upon the CRC to prevent the use of partial packets. If the 
CRC fails one time in 64K packets, at about 30 packets/ 
second, every few thousand seconds there is a potential 
undetected error. When a broken-packet CRC fails, it fails 
every time the packet is used. This means that a failure that 
produces a click may end up reproduced every time the same 
splice is made. This requires the use of a garbage-collecting 
process to remove broken audio frames. 

A second approach to splicing audio is to include "unfin- 
ished" from-stream audio frames (i.e., those overlapping a 
video splice) in the resultant stream. The to-stream audio 
frames are then retimed such that they butt up against the 
"unfinished" from-stream audio frame. This technique 
assures continuous audio at the expense of continuous 
inspection, buffering and adjustment of audio frames and 
packets. The first complete to-stream audio frame is selected 
as the first audio frame to play because the to-stream frame 
which has already started is likely to be delayed too much to 
match the end of the "unfinished" from-frame. This tech- 
nique also causes a slightly distorted lip- sync. 

A third approach to splicing audio is to maintain align- 
ment of audio with its corresponding video, that is, to leave 
a gap between the end of the from-stream audio frame and 
the beginning of the to-stream audio frame. This approach 
advantageously relies on the MPEG decoder specification 
which requires that audio frame gaps are muted. In this 
approach the audio presentation-time -stamps are adjusted by 
the same adjustment amount applied to the video frames. In 
this manner lip sync is maintained. The third approach is 
especially useful when many splices (e.g., the creation of a 
sequence from a succession of short splice segments) may 
produce audio artifacts due to the muting. 

FIG. 5 depicts a plurality of splicing scenarios involving 
audio alignment which illustrate aspects of audio -video 
splicing, assuming the above cited third approach is used to 
maintain alignment of audio with its corresponding video. 

FIG. 5A depicts the simple splicing case where both audio 
streams align with their corresponding video. The splicer 
delays both to -streams and simply switches at the splice 
point. 
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FIG. SB depicts the splicing case where the from-streara of splice points. Such header information comprises, e.g., a 

video and from-stream audio are aligned, but the to-stream splicing_point_flag, a splice_type field and a splice_ 

video and to-stream audio are not aligned. When the splice countdown field. The splicing_point flag, when set to 1, 

is made, a partial to-stream audio frame is discarded. The indicates that a splice__countdown field is present. The 

next complete to-audio frame is passed to the output with 5 S plice_countdown field is an 8-bit integer specifying the 

appropriate delay. remaining number of transport stream packets remaining 

FIG. 5C depicts the typical splicing case where both audio unt il a splicing point is reached, such as the end of a video 

streams do not align with the corresponding video streams. fr ame -t^ S plice_type field is a 4-bit field used to derive 

It can be seen that a from-stream audio frame has already splice__decodine_delay and max_splice_rate data from, 

begun before the splice point. This audio-frame is buffered 10 a toble stori such data ^ standard use the these 

and transferred to the output. It doesn t end until some headef fl and ^ tQ { lemem a lici function is 

fraction lof a frame time after the splice. The to-stream audio defined ^ Mp£G 

frame that spans the splice-point cannot be used. The next r 

to-stream audio frame also cannot be used. It begins too ^ previously discussed, splice points within a transport 

early, and would overlap the last from-stream audio frame. stream may be in-points, out-points or both. An out-point is 

The first to-stream audio frame that appears in the output 15 equivalent to the MPE-definition of a splice point. An 

stream begins D time units after the splice-point. This delay in-point comprises a splice point (i.e., an out-point) followed 

may be as much as two audio frames. by a sequence header that is immediately followed by an 

In the above example (FIG. 5C) the lip-sync is preserved, I-frame. Therefore, in-points within a particular stream 

but as much as 32 mS of from-stream audio overlaps the maybe identified by finding out-points followed by sequence 

to-stream video. Also, the first to-stream audio begins as late 20 headers that are followed immediately by I-frames. Thus, a 

as 64 mS after to-stream video begins. Finally, the splicer to-stream may be entered at any in-point, as described 

performing the splicing operation must buffer a whole audio above, even if the out-point of a from-stream is not followed 

frame in each work buffer. by a sequence header or an I-frame. In this embodiment, the 

Ideally, every audio frame includes a PTS. It is possible splicing_point_flag of the packet immediately preceding 
that some equipment manufacturers only include a PTS 25 t h e out-point must equal one, and the splice_countdown 
every, e.g., third audio frame. In this case, or the case where field of that packet must equal zero. However, the above- 
there is no audio PTS, a splicing operation may be per- described embodiment requires that the bitstream be parsed 
formed after calculating a virtual time-stamp. The virtual down t0 the elementary layer to examine, e.g., the picture^ 
time stamp is derived from the approximate real-time delay codmg _jv pe field in the picture header to determine if an 
oi audio-frames from video reference time-stamps. The 30 j.f rame ^ present 
virtual audio-time-stamp is then incremented by the (known) 

audio-frame duration on successive audio-frame starts. This In the preferred embodiment of the invention, entrance 

process may be implemented as a backup process to ensure and exit indicia comprise information residing within the 

that non-time stamped audio streams to not get into a studio transport layer, thereby obviating the need to parse the 

where time stamps are crucial to the operation of the studio. bitstream down to its elementary layer. Specifically, an 

Auxilia Data S licin out-point in a from-stream is indicated by the splicing__ 

y P & point_flag being equal to one and the splice_countdown 

Auxiliary data is included in many MPEG streams. This field being equal to zero. Similarly, an in-point in a to-stream 

data is usually present as contiguous, unbreakable streams of ^ indicated by the splicing_point_flag being equal to one 

unknown length. By contrast, a compressed audio stream is and tne S plice_countdown field being equal to negative one. 

relatively well-behaved and predictable. Tlie auxiliary data 40 aQ in _ pomt packet ^ the packet [hat 

stream may be associated with a corresponding video stream follows an in-point) and an out-point packet (i.e., the packet 

having a delay that is not now specified. including an out-point) may be the same packet. Moreover, 

There are several methods for handling auxiliary data in the splice_type of an in-point packet (i.e., the packet that 

a splicing operation, including: 1) ignore it and splice the immediately follows an in-point) indicates the suitability of 

auxiliary data at the same time as the video data; 2) insert 45 splicing the in-packet to an out-packet in that the in-packet 

auxiliary data through a separate path to, e.g., a play-to-air and the out-packet should both have the same splice_type. 

switcher (this data _ may comprise z program guide or other In this embodiment , an out . p oint that is not also an 

consumer-related information); and 3) define a set of seg- m int must have ^ splicing _ poin t_fl a g equal to zero in 

mentation markers for auxiliary data and rely upon these me ket immediatclv following he packet with the 

markers in switchers to keep segmentation correct (this 50 n& _ point _ flag ^ equal to one and the splice__ 

requires knowledge about the content of auxiliary data countdown being equal to zero . In this mannef} the 

streams and the lengths of auxiliary data segments within S plkingjoinLJiag indicates that the out-point is not asso- 

those streams) Hie auxiliary data may also be switched with ciated with an in ^ ^ ^ because the splic ing_point_ 

or without a delay, and the delay may be a parameter passed fl must be { {Q QnQ for the of the u 

to the switcher by a decision making controller. The auxil- 55 coumdown field t0 be y ^ afld ^ splice _ COUDtdown field 

lary data may be input to the sphcer via an auxiliary input must be vaHd afld ^ tQ ^ on£ for m { [ni 

Input arriving on the auxiliary-data input may be buffered , , , . ° 

and inserted into the output stream on a space-available . The above described embodiment of indicia for identify- 

basis as a replacement for null packets. In this case it m f . m c ?^« mdicia) and out-pointe (i.e., exit 

becomes some other system unit's responsibility to align *° in , dicia > wlXh } n an MPEG-like bitstream provides several 

such data within streams, and to provide channel capacity ^vantages One advantage, as previously described, is the 

for inserted aux-data by, e.g., reducing a video stream data ^My to identify in-points and out-points by parsing only 

ratc the adaptation header of the transport layer, and avoiding the 

parsing of the elementary layer. 

IQ-Point and Out-Poinl Syntax and Usage 6S Mothe[ ^vantage of the preferred embodiment is that 

Various header fields within the adaptation header of each transport packet is self contained in the sense that each 

transport stream packets are used to indicated the presence packet contains sufficient information to determine if the 
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particular packet comprises an in-point, an out-point or both. 
This allows the identification of an in-point or out-point in 
a packet without regard to any previous packets. Thus, there 
is no need to, e.g., compare the splice_type field of a 
probable in-point packet to the splice_type field of the 
preceding packet before determining that the probable 
in-point packet is, in fact, an in-point packet. However, it 
must be noted that the splice„type field indicates the 
suitability of splicing the particular stream and another 
stream associated with another splice_type field. 

As previously mentioned with respect to bitstream 
generation, it is desirable to generate bitstreams according to 
the in-point and out-point syntax described above. 
Specifically, it is desirable to include entrance and exit 
indicia at many places within the bitstream, illustratively, at 
every I-frame during the bitstream encoding process. To 
accomplish this inclusion of entrance and exit indicia, the 
transport multiplexer used to generated the transport stream 
to be subsequently spliced must be provided with informa- 
tion identifying packets associated with, e.g., I -frames. 
Alternatively, an elementary stream parsing to identify 
appropriate entrance and exit points within a stream may be 
implemented. 

FIG. 6 depicts a flow diagram of a routine 600 suitable for 
identifying in-points and out-points in accordance with the 
invention. Specifically, the routine 600 is suitable for use by, 
illustratively, the bitstream examiners 310A and 310B of the 
splicer 300 depicted in FIG. 3. The routine 600 is entered at 
step 605 when a transport packet within a stream to be 
examined (e.g., S6 or S7) is received by, e.g., a bitstream 
examiner (e.g., 310A or 310B). The routine 600 proceeds to 
step 610, where the packet header of the received packet is 
examined, and to step 615, where a query is made as to 
whether the splice_point__flag within the adaptation header 
of the received packet is equal to 1. 

If the query at step 615 is answered affirmatively, the 
routine 600 proceeds to step 620, where a query is made as 
to whether the splice_countdown flag is equal to 0. If the 
query at step 620 is answered affirmatively, then the routine 
proceeds to step 635, where the packet is identified as 
containing an out-point. Such identification may take the 
form of setting an "out-point-ready" flag suitable for use in, 
e.g., step 210 of the routine 200 of FIG. 2. The routine then 
proceeds to step 635, where it is exited. 

If the query at step 620 is answered negatively, then the 
routine 600 proceeds to step 630, where a query is made as 
to whether the splice_countdown flag is equal to -1. If the 
query at step 630 is answered affirmatively, then the routine 
proceeds to step 640, where the packet is identified as 
containing an in-point. Such identification may take the 
form of setting an "in-point-queued" flag suitable for use in 
step 208 of the routine 200 of FIG. 2. The routine then 
proceeds to step 645, where it is exited. 
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examining a valid splice__countdown field within a por- 
tion of a transport packet; wherein: 
said entrance indicium is identified by said splice 

countdown field being equal to a first value; and 
said exit indicium is identified by splice countdown 
field being equal to a second value. 

2. The method of claim 1, wherein said first value is 
negative one and said second value is zero. 

3. The method of claim 1, wherein said splice countdown 
field is only valid if a splice point flag within said portion of 
said transport packet is equal to a third value. 

4. The method of claim 3, wherein said third value is one. 

5. A method for identifying one of entrance indicium and 
exit indicium in an MPEG-like transport packet, said 
entrance indicium being associated with a transport packet 
suitable for use as a first packet in a splicing segment, said 
exit indicium being associated with a transport packet suit- 
able for use as a last packet in a splicing segment, said 
method comprising the steps of: 

examining a splice point flag within a header portion of 

said transport packet; and 
in the case of said splice point flag being set to a first 
value, examining a splice countdown field in said 
header portion of said transport packet; wherein: 
said entrance indicium comprises said splice count- 
down field being equal to a second value; and 
said exit indicium comprises said splice countdown 
field being equal to a third value. 

6. The method of claim 5, wherein said first value is one, 
said second value is negative one, and said third value is 
zero. 

7. In a system for generating an MPEG -like transport 
bitstream in response to one or more elementary streams, a 
method for including one of an entrance indicium and exit 
indicium in one or more MPEG-like transport packets, said 
entrance indicium being associated with a transport packet 
suitable for use as a first packet in a splicing segment, said 
exit indicium being associated with a transport packet suit- 
able for use as a last packet in a splicing segment, said 
method comprising the steps of: 

identifying packets suitable for use as first or last packets 
in a splicing segment; and 

setting to a first value a splice point flag within a header 
portion of each of said identified packets to include said 
entrance or exit indicium; and 

setting to a second value a splice countdown field within 
said header portion of said identified packets to include 
said entrance indicium, and setting to a third value said 
splice countdown field within said header portion of 
identified packets to include said exit indicium. 

8. The method of claim 7, wherein said first value is one, 
said second value is negative one, and said third value is 
zero. 

9. The method of claim 7, wherein said entrance indicium 



Anu , - , , . . , . t iL is associated with a packet the precedes a sequence header 

Although various embodiments which incorporate the 55 that immediately precedes, in the case of a video stream, an 

I-frame. 

10. The method of claim 8, wherein said entrance indi- 
cium is associated with a packet the precedes a sequence 
header that immediately precedes, in the case of a video 

60 stream, an I-frame. 

11. A method for splicing bitstreams, said bitstreams 



teachings of the present invention have been shown and 
described in detail herein, those skilled in the art can readily 
devise many other varied embodiments that still incorporate 
these teachings. 
What is claimed is: 

1. A method for identifying one of entrance indicium and 
exit indicium in an MPEG-like transport packet, said 
entrance indicium being associated with a transport packet 
suitable for use as a first packet in a splicing segment, said 
exit indicium being associated with a transport packet suit- 
able for use as a last packet in a splicing segment, said 
method comprising the steps of: 



comprising a plurality of information segments representa- 
tive of a sequence of information frames, said method 
comprising the steps of: 

monitoring a first bitstream to detect an exit indicium, said 
first bitstream being coupled to an output and including 
at least one of said exit indicium, said exit indicium 
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indicative of an appropriate last information segment of 
a splicing segment; 

providing a second bitstream, said second bitstream 
including at least one entrance indicium, said entrance 
indicium indicative of an appropriate first information 5 
segment of a splicing segment; and 

coupling said second bitstream to said output in response 
to a control signal, a detection of said exit indicium in 
said first bitstream and a detection of said entrance 
indicium in said second bitstream; wherein 
said exit indicium comprises a valid splice countdown 
field in said header portion of said information 
segment being set to a first value. 

12. The method of claim 11, wherein said first value is 
zero. 

13. The method of claim 11, wherein said entrance 
indicium comprises a valid splice countdown field in said 15 
header portion of said transport packe being set to a second 
value. 



,000 

24 

14. The method of claim 13, wherein said second value is 
negative one. 

15. The method of claim 11, wherein said step of moni- 
toring said first bitstream comprises the steps of: 

examining a splice point flag within a header portion of 

each information segment; and 
in the case of said splice point flag being set to a third 
value, examining said splice countdown field in said 
header portion of said information segment, said 
entrance indicium comprising said splice countdown 
field being equal to a second value. 

16. The method of claim 14, wherein said second value is 
negative one. 

***** 
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